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ABSTRACT 

There is a need to explore methods for reducing 
lengthly computer turnaround or clock time associated 
with engineering design problems. Different strategies 
can be employed to reduce this turnaround time. One 
strategy is the use of a supercomputer, which can be cost- 
ly in terms of hardware acquisition and software 
modification. Another strategy is to run validated 
analysis software on a network of existing smaller com- 
puters so that portions of the computation can be done in 
parallel. This paper focuses on the implementation of this 
second strategy using two types of problems. The first 
type is a traditional structural design optimization 
problem, which is characterized by a simple data flow 
and a complicated analysis. The second type of problem 
uses an existing computer program designed to study 
multilevel optimization techniques. This problem is 
characterized by complicated data flow and a simple 
analysis. The paper shows that distributed computing can 
be a viable means for reducing computational turnaround 
time for engineering design problems that lend themsel- 
ves to decomposition. Parallel computing can be ac- 
complished with a minimal cost in terms of hardware and 
software. 

INTRODUCTION 

Traditionally, large aerospace design problems are 
divided into disciplines with each discipline contributing 
to the design of one or more components and to the con- 
figuration of the entire vehicle. This division of labor 
simplifies the task of individual design teams and allows 
them to make progress even when physically separated 


from one another. The division of large design problems 
into subsystems also avoids some of the limitations that 
computers have in handling the total structural analyses 
and design optimization of large problems. 

In recent years there has been promising research 
done in the field of multilevel optimization for large 
structural design problems . 1,2 Using multilevel optimiza- 
tion methods, large problems are decomposed into hierar- 
chically related smaller subsystems, where the structural 
analysis and design optimization for each subsystem are 
done simultaneously. 

The multilevel optimization method relates well to 
current research in computer science in the areas of 
parallel processing 3 on a parallel computer and distributed 
processing over a network of computers. If optimization 
for the subsystems can be run simultaneously, then turn- 
around time for the total system optimization can be 
reduced. While parallel processing is a means of im- 
plementing multilevel decomposition techniques, this op- 
tion generally requires hardware acquisition and computer 
code modification. A less costly option is to use an exist- 
ing network of computers to simulate parallel processing. 
It is, therefore, the purpose of this paper to investigate 
distributed computing over a network of existing worksta- 
tions as a means of decreasing the turnaround time for 
design optimization problems. Distributed computing will 
be applied to two types of problems. The first problem 
chosen is a traditional structural design optimization 
problem, which is characterized by simple data flow and 
complicated analysis. The second problem chosen is an 
idealized design optimization problem that utilizes multi- 
level optimization techniques. This problem is charac- 
terized by complicated data flow and simple analysis. 
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AVAILABLE COMPUTER RESOURCES 

The hardware chosen for this project is a network of 
eight MicroVAX workstations and an Ethernet local area 
network(LAN) circuit (Figure 1). Each Micro Vax 
workstation is configured with 6 million bytes (6MB) of 
main memory and a 71MB hard disk. 

Although eight workstations are available for dis- 
tributed computing, only a subset of these is used. This is 
done for several reasons. First, experience has shown that 
hardware problems occur and requiring all eight of the 
workstations to be operational at all times is unrealistic. 
Second, the primary user of a workstation often has a 
CPU intensive program running, which inhibits the execu- 
tion of the distributed batch processes. Therefore, the 
workload for a workstation is examined before it is 
chosen to be used in the distributed system. 



Figure 1. Network of workstations 


The workstations are connected by two networks, 
DECnet and LaRCNET. DECnet is the product name for 
software and hardware that allows Digital Equipment 
Corporation (DEC) computers to operate over a com- 
munications network. LaRCNET 4 is a local area net- 
work developed at the NASA Langley Research Center 
(LaRQ to provide file transfer between multi-vendor dis- 
tributed computers at the Langley site. Both networks 

*Use of commercial products and names of manufacturers in 
this report does not constitute an official endorsement of 
such products or manufacturers, either express or implied, 
by the National Aeronautics and Space Administration. 


use the same physical Ethernet, but because of software 
implementation differences in accessing the network, 
LaRCNETs transmission rate is faster than DECnet’s. 
However, because LaRCNET sometimes experiences 
problems and is not operational, a choice of network is 
made at execution time. The software selected for this 
project consists of application programs written in 
FORTRAN 77, and the DEC VAX/VMS 5 operating sys- 
tem, which allows the use of remote batch processing. 
The two application programs are described next in this 
paper. 

DISTRIBUTED STRUCTURAL 
OPTIMIZATION 

In 1986 engineers at Langley Research Center were 
given the task of redesigning the solid rocket booster 
(SRB) joint that had failed in the Shuttle tragedy. One 
group of engineers at LaRC was involved in optimizing a 
new design that might be a candidate for the booster joint 
if the existing SRB joint could not be successfully 
modified 6 ’ 7 . 

The SRB joint redesign is a structural sizing problem 
whose goal is to reduce the weight and stresses in the 
joint PROSSS 8 , a system of structural analysis and op- 
timization computer programs is used in this design op- 
timization work which is done on a DEC MicroVAX 
workstation. Seven design variables are used in the op- 
timization process. To estimate the gradients of the ob- 
jective function and constraints with respect to a given 
design variable, that variable is perturbed and the finite 
element model is reanalyzed. The analysis of the finite 
element model is repeated eight times, once for each 
design variable and once for the baseline. A single op- 
timization cycle (seven design variables and the baseline 
analysis) takes three hours and forty five minutes to com- 
plete. The time spent by the optimization software is 
negligible compared to the time spent in analysis. Since 
five to seven cycles are needed to converge to final op- 
timization results, the turnaround time is from nineteen to 
twenty six hours. Because of the need to reduce turn- 
around time, it is expedient to distribute the SRB joint 
design optimization system over a network of worksta- 
tions. 

Since all eight of the workstations used in this project 
are seldom available at one time, the system is distributed 
to some set of four. Figure 2 shows the mapping of the 
problem onto four workstations. Three of these worksta- 
tions run only the analyses for the perturbed design vari- 
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modifications are required in PROSSS to allow the 
analysis for the design variables to be distributed. 


The parallel processing for this distributed system is 
initiated by a set of five VAX/VMS DEC command lan- 



Figure2. Four workstation distribution for 
SRB joint opt 


guage (DCL) files: one interactive procedure and four 
batch procedures. The interactive procedure queries the 
user for input parameters such as: total number of design 
variables, maximum number of optimization cycles, 
names of the workstations to use, which network software 
to use, and the time limit to wait for the data file to be 
received. The interactive procedure also submits the 
batch procedure files to queues on the distributed 
workstations. The four batch procedures control the data 
flow between the workstations. Their purpose is to 
monitor data files, execute the analysis, and then send the 
results to the next step of analysis. These batch proce- 
dures also contain instructions to close down the dis- 
tributed system if a data file does not arrive within a 
specified time limit These checks are necessary because 
of the possibility of hardware and software problems. It 
is observed that if one of the workstations procedures ter- 
minates abnormally, then the other workstations wait in- 
definitely and large event logging files are generated 
which eventually fill these workstation disks. 

This design problem has the necessary computational 
requirements (long analysis times and uncomplicated data 
flow) to demonstrate the value of a distributed computer 
system. The resulting distributed system reduces the time 
of one optimization cycle from three hours and forty five 
minutes to one hour. This work is documented in 
reference 6. 

DISTRIBUTED MULTILEVEL 
OPTIMIZATION 

As aerospace systems become more complex, the 
need for automatic and mathematically rigorous system 


integration becomes more critical. Multilevel optimization 
is one technique for integrating component designs into a 
total system design and then iteratively improving the 
performance of the system as a whole. Multilevel op- 
timization can be hampered by the sheer size of the 
problems involved. For example, optimizing the design 
of a SRB joint component taxes micro-computer resour- 
ces; combining all design components into an optimum 
solid rocket booster model can overwhelm the largest 
computer available. 

Application of distributed processing is another way 
to improve the performance of system integration techni- 
ques. Typically, the most complicated system, with 
hundreds of components and thousands of design vari- 
ables, has relatively small amounts of data coupling. This 
means that the components can be designed separately 
and in parallel. Only gross details and the sensitivity of 
those solutions to changes in other parts of the system 
must be communicated. Thus, the same type of parallel 
processing which is so effective in the SRB joint design 
should be beneficial in large design problems. 

The Multilevel Simulator 

The multilevel simulator is a computer program 
designed to investigate multilevel optimization techni- 
ques. The simulator mimics the qualitative behavior and 
data couplings occurring among subsystems of a complex 
engineering system. Simple analytical functions are used 
in place of realistic disciplinary analyses. In this way, 
numerous multilevel optimization techniques can be in- 
vestigated in a relatively short period of time. 

Converting the multilevel simulator for distributed 
processing is considerably more complicated than for the 
SRB joint design problem. System design problems often 
decompose into numerous levels. For example, an 
aircraft is composed of different subsystems such as 
propulsion, wings, fuselage, etc. The propulsion subsys- 
tem is also a collection of subsystems such as compressor 
and turbine. Thus, the aircraft system has at least three 
levels. The multilevel simulator allows decomposition to 
any number of levels with any number of subsystems on a 
level. 

The original multilevel simulator is a single 
FORTRAN computer program. Information is passed 
from one subsystem to another as parameters in sub- 
routine calls. The distributed multilevel simulator re- 
quires the conversion of subroutines into distributed com- 
puter programs. Minor software changes are required in 
the input and output sections of the distributed programs. 
Each subsystem needs to receive data from a lower level, 
execute its optimization code and then pass the computed 
results to the next higher level. In addition to changes in 
the simulator programs to allow for input and output of 
data, one new FORTRAN program is needed to act as a 
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data manager. The data manager collects the data from 
the subsystems on one level, and then transfers the com- 
bined data to each of the subsystems on the next level. 
Each level has a continuously executing data manager 
program. 

The test case chosen has 25 design variables and is 
decomposed into four levels with a total of nine units 
(one system and eight subsystems) (figure 3). To allow 
the work of the simulator to be distributed to different 
processors requires the partitioning of the multilevel 
simulator application program. Nine copies of the op- 
timization code and related analysis codes are required for 
the test case. The distributed applications codes are 
denoted by TOPLEVEL and LEViSj, where i denotes 
level number and j denotes subsystem number. Thus 
LEVIS 1 is the code for Level 1 subsystem 1, LEV2S1 is 
the code for Level 2 subsystem 1 and so on. 

MULTILEVEL DECOMPOSITION FOR TEST CASE 



Figure 3. Schematic of multilevel simulator 

Five workstations are used for distributing the subsys- 
tems (figure 4). Of the five workstations used, one is the 
controller workstation on which the system level and data 
manager codes are executed and four handle the sublevels 
and their analyses. Each workstation handles all levels 
for the same subsystem. For example as shown in 
Figures 3 and 4, LEV1S3, LEV2S3, and LEV3S3 are 
grouped together on the same workstation. This is 
facilitated in VAX/VMS by making a directory or 
separate work area for each level on the workstation. 
With the five workstation hardware configuration, any 
number of levels can be handled with the number of sub- 
systems per level limited to four. 

DCL Command Files 

After partitioning the FORTRAN application program 


into independent program units, the next task is to adapt 
the command files that distribute and control the execu- 
tion of the system. These command files are written in 
the VAX/VMS DEC Command Language (DCL) 5 . There 
are DCL command files for each subsystem (8), each data 
manager (2) and the total system (2), resulting is a total 
of twelve for the test case. Of these twelve DCL files, 
there are five types (see Appendix for a listing). Type I is 
an interactive procedure and Types II, III, IV and V are 
batch processors that are started by the interactive proce- 
dure and continue to execute until a termination file ap- 
pears in their directory or work area. 

The interactive procedure, Type I, is similar to the in- 
teractive procedure for the distributed SRB joint problem. 
It queries the user for input parameters as to which net- 
work to use, the time limit that a workstation should wait 


for data to arrive, number of levels and subsystems, 
which workstations are to be used in the distribution, and 
which workstation is acting as the controller. This com- 
mand file also distributes the FORTRAN executable file, 
input data file and a DCL command file to each worksta- 
tion for each subsystem executing on that workstation. 
Its final function is to submit the DCL command file to 
the batch queue of each distributed workstation for execu- 
tion. 

Type II is a batch procedure file that waits for the re- 
quired output data files from the other subsystems on that 
same level, executes the data manager for that level, and 
then passes the combined data up to the subsystems on 
the next level as input files. There are two such proce- 
dure files, one for level 2 and one for level 3. 

A Type III procedure file executes each subsystem 
and is the simplest of the three procedure files. Its func- 
tion is to wait for a data input file, execute the optimiza- 
tion code and pass the output to the level data manager. 



Figure 4. Five workstation distribution for 
simulator 
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the results down to the lowest level. If convergence has 
occurred or the maximum number of cycles has been met, 
the lop level optimizer generates an output file. This out- 
put file acts as a signal to the time checker procedure 
(Type V) to send the termination file to all workstations. 

Type V is a procedure that is used as the time check- 
er and system terminator. The interactive procedure 
(Type I) passes the time limit, and workstation names to 
this procedure as parameters. When the time limit is ex- 
ceeded or the top level optimizer generates an output file, 
a termination file is sent to all workstations. 


Table 1 


Problem #Level^Design 

Variables 

Turnaround 
in min 

File 

Transfer 

Simulator 
Distributed 4 

25 

7 

33 

Simulator 
Sequential 4 

25 

2 

0 

SRB Distributed 1 

7 

60 

18 

SRB Sequential 1 

7 

225 

0 


RESULTS 


Table 1 summarizes the difference between the dis- 
tributed and sequential version of both the SRB joint 
redesign and multilevel simulator problems. Notice that 
the turnaround time for the distributed simulator is longer 
than that for the original sequential program. That is to 
be expected because the simulator test case has many 
levels, many design variables and lots of files to transfer, 
but has very simple analysis requirements. The data 
transfer requirements for the distributed simulator test 
case simply overwhelm the problem and explain why the 
distributed case takes over three times more clock time 
than the sequential case. If this was an actual system 
design problem instead of a simulated one, the cost of 
data transfer would probably be negligible compared to 
the cost of analysis. For example, utilizing distributed 
computing in a component design problem like the SRB 
joint design produces considerable savings in turnaround 
time. Here the number of levels and design variables is 
small, the number of files transferred is moderate, but the 
analysis time is large. 

CONCLUDING REMARKS 

It is concluded that using a distributed computer sys- 
tem of existing hardware, linked by a network, is an ef- 
fective way to reduce turnaround time. The benefits of 
parallel computing can be simulated with minimal chan- 


ges to application software. Experimentation with the 
distributed and serial implementations of the multilevel 
simulator reveals that the time required for data transmis- 
sion from one network workstation to another may exceed 
the total turnaround time savings if the analyses per- 
formed at each workstation have short execution times. 
On the other hand, the distributed implementation of the 
SRB joint problem proves to be beneficial because the 
analysis time is extensive. Thus, the conclusion is reached 
that the distributed system is workable and will reduce 
turnaround time for typical engineering design problems. 
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APPENDIX 

This Appendix contain* the DCL procedure files used to implement the distributed 
system far the multilevel sim ulster described in this pspex. Figure 3 is a 
schomslic of the 25 variable test case and figure 4 is a mapping of this test case 
onto five Micro VAX computers. 

Each subsystem is initialized with the following files: 

LEViSj.IN input data (constant throughout test case) 

LEViSj.EXE FORTRAN optimization code 

RUNij DCL command file 

V.DAT input parameters 

Each subsystem communicates with the data manager using the following files: 
LEViSj.CM input data from data manager 

LVij.CM output data sent to data manager where i denotes level number 

and j denotes which subsystem on that level 

The DCL commands are of the five types defined below. Although a choice of 
network option is used in the test case, only DECnet commands are included here 
far simplicity. 

TYPE I Interactive procedure for total system 

$! PROCEDURE TO DISTRIBUTE JOBS AND DATA TO WORKSTATIONS 
$! WRITTEN IN DEC COMMAND LANGUAGE (DCL) 

$! 

$ ON ERROR THEN GOTO ERR 
S! 

$! INITIALIZE AND INQUIRE ABOUT INPUT PARAMETERS 
$ INQUIRE TIMEL " ENTER TIME LIMIT FOR THE PROCESSES IN 
MINUTES” 

St 

St LEVEL AND subsystem INFORMATION 

S! 

S INQUIRE NL "PLEASE ENTER NUMBER OF LEVELS" 

S INQUIRE NUML3 "ENTER NUMBER OF SUBSYSTEMS IN BOTTOM 
LEVEL" 

S INQUIRE NUML2 "ENTER NUMBER OF SUBSYSTEMS IN NEXT LEVEL" 

$ INQUIRE NUML1 "ENTER NUMBER OF SUBSYSTEMS IN TOP LEVEL" 

St 

$! DETERMINE MAXIMUM NUMBER OF SUBSYSTEMS ON A LEVEL 
St 

$ MAXS - NUML1 

$ IF NUML1 LT. NUML2 THEN MAXS - NUML2 
$ IF NUML3 GT. MAXS THEN MAXS - NUML3 

St 

$! SELECT WORKSTATIONS 
St 

$ INQUIRE N1 "ENTER WORKSTATION NAME FOR THE DATA 
MANAGERS" 

$ NCT * 1 
S REPEAT: 

SNCT-NCT + 1 

$ INQUIRE N’NCT "PLEASE ENTER WORKSTATION FOR SUBSYSTEMS 

S IF NCT LE. MAXS THEN GOTO REPEAT 
St 

S! CLEAN UP FILES ON WORKSTATIONS 
S! 

S DEL [SIM]* OUT;* 

$ DEL [SIM....]* OUT;* 

$ DEL [SIM..]* CM;* 

$! 

S! COPY DATA FILES AND DATA MANAGERS TO WORKSTATION 
St 

S COPY V.DAT [SIM.LEVEL3]V.DAT 
S COPY DATA.MANAGER.EXE [SIMLEVEL3]* 

S COPY DATA _ MANAGER. EXE [SIMLEVEL2]* 

S COPY TOPLEVEXE [SIM LEVEL 1 ]* 

$ COPY RUNC23.COM [SIM.LEVEL3JRUNC23.COM 
S COPY RUNC12.COM [SIM.LEVEL2]RUNC1 2.COM 
S COPY RUNT.COM [SIM.LEVEL1JRUNT.COM 
S! 

S! COPY FILES FOR SUBSYSTEMS TO EACH LEVEL OF WORKSTATION 
S! 

S LEVEL1 - 0 


5 LEVELCT - 0 
S LEVELCYCLE: 

S LEVELCT - LEVELCT + 1 
S IF (LEVEIXT.GT.NL) THEN GOTO CONTINUE 
$ SUBLIM - NUML’LCT’ 

$ SUBS YSTEM.COUNT - 0 
SCT-2 

$ LEVEL 1 » LEVEL1 ♦ 1 
S SUBSYSTEMCYCLE: 

$ SUBSYSTEM.COUNT - SYBSYSTEM.COUNT + 1 
$ IF (SUBSYSTEM_COUNT.GT.SUBUM) THEN GOTO LEVELCYCLE 
S WORKSTATION - N CT’ 

S DEL WORKSTATION’ ::[SIM.LEVEL’LEVEL1 '} V.DAT;* 

S DEL ’WORKSTATION’ ::[SIM.UEVEL’LEVEL1 ’j*.CM;* 

$ FILEl - FSFAO("!SL!SL"X£VELLSUBSYSTEM_COUNT) 

$ FILE2 = F$FAO("! AS!SL!SL!AS","RUN",LEVEL1, subsystem coum/.com") 

S FILE 3 - 

F$FAO("!AS!SL!AS!SL!AS" > "LEV"X£VELl,"S",subs ystem_count,".EXE") 
SFILE4- 

F$FAO("!AS!SL! AS!SL!AS", "LEV" X£VELL,"S", subsystem count,". IN") 

S! USE DECNET 

SI 

$ C0PY1: 

SCOPY ’FILE2’ ’WORKSTATION’ :: [SIM. LEVEL’ LEVEL!*]* 

SCOPY ’FILES’ ’WORKSTATION’ : :[ SI M.LEVEL’ LEVEL 1 ’]* 

SCOPY ’FILE4* ’WORKSTATION’ ::[SIM.LEVEL’LEVELrj* 

S IF LEVEL 1 NE. NL THEN GOTO NEXT 
S COPY V.DAT ’WORKSTATION’ ::[SIM.LEVEL’LEVEL1 ’]* 

S NEXT: 

$! 

S! SUBMIT COMMAND FILE TO BATCH QUEUE 
S! 

$ SUBMIT/REMOTE ’WORKSTATION’:: [SIM. LEVEL’ LEVEL 1 ’JRUN’FILEl ’ 
-/PARA METERS -(’NU) 

$CT*CT+ 1 

$ GOTO subaystemCYCLE 

$! 

$! SUBMIT DATA MANAGER COMMAND FILES TO BATCH QUEUE 

S! 

$ CONTINUE: 

SI 

$ IF NL EQ. 2 THEN GOTO LEVEL2 
S SUBMIT/NOPRINT [SIM.LEVEL3JRUNC23 
- /PARAMETERS-(’NUML3’,’NUML2 , ,’N2 , , , N3’,’N4 , .’N5 , > ) 

$ LEVEL2: 

$ SUB MIT/NO PRINT [SIM.LEVEL2JRUNC 1 2 - 
/PARAMETERS-(*NUML2 , t ’NUMLr,’N2 , i ’N3’, , N4’, , N5 , l ) 

S SUB MIT/NO PRINT [SIM.LEVEL1 JRUNT - 
/PARAMETERS<NUML1’,’NUML3’/NL , ,’N2’ 1 , N3’,’N4*;N5’) 

S SUBMIT/NOPRINT [S IM. LEVEL 1JTIMECK - 
/PARAMETERS-CTIMEL’ , ’N2’ , ’N3 ’N4 ’ , ’N5’) 

$ GOTO FINISH 


SERR: 

$ WRITE "YOU HAVE AN ERROR IN THIS PROCEDURE' 
$ FINISH: 


TYPE II Batch file for data managers 


$! 

$! RUNC12.COM - DATA MANAGER BETWEEN LEVELS 1 AND 2 
S! 

$S INPUT 

$! PI - # OF SUBSYSTEMS IN LEVEL 2 

S! P2 = # OF SUBSYSTEMS IN LEVEL 1 

$! P3 « NODE NAME FOR LEVEL1 SI RUN 

5! P4 = NODE NAME FOR LEVEL! S 1 RUN 

S! P5 - NODE NAME FOR LEVEL1 S3 RUN 

S! 

S! CLEAN UP FILES AND INITIALIZE 
S! 

S SET DEF [SIM.LEVEL2] 

$ DEL OUT.DAT;* 
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$ PURGE *.* 

S SET VER 
$ START1 : 

$ VNUM-2 
$ LSUB - PI 
$ NSUB - 0 
$ FNUM * 20 

$1 BEGIN WAITING LOOP FOR INPUT DATA FILES 
$! 

S IF LSUB .EQ. 0 THEN GOTO CHECKV 
$ LSUB - LSUB - 1 
SFNUM -FNUM+ 1 

ji CONSTRUCT INPUT FILE NAMES 

$ UNIT - FJFAOC'AS!SL!AS","LVTNUM,".CM ) 

S START* 

$ SET MESSAGE/NOF/NOS/NOI/NOT 

$ NOTOPEN1 : 

si CHECK FOR TERMINATION FILE OUT.DAT AND INPUT DATA FILES 

S’. 


S FILEX * F$ FSSEARCHfOUT.DAr) 
SffSEX .NES. - THEN GOTO FINISH 


$ O^/S/BRROR-NOTOPENl FILE ’unit 

$ CLOSE FILE 
$ GOTO BEGIN 

$! CHECK FOR INPUT FILE V.DAT 
S! 

S CHECKV: 

S FILEX - FS SEARCH ("OUT. DAT") 

$ IFFILEX .NES. " THEN GOTO FINISH 
$ WAIT 00:00:05.00 

$ OPEN/READ/ERROR-CHECKV FILE V.DAT 
$ CLOSE FILE 
$ SET MESSAGE/F/S/I/T 

$i RUN DATA MANAGER PROGRAM 
$! 

$ RUN DAT A_MANAGER 

si SEND COMBINED DATA UP TO SUBSYSTEMS ON NEXT LEVEL 

S! 

$ CT = 3 
$ SEND _D AT A: 

$ WORKSTATION =P’CT* 

$ S LV^! S-^aUl! AS-.-LEVEL1S WOT 
t TOPY TVS’ ’WORKSTATION’ ::[SIMTEVEL1 ] LVS 

SCOPY V.DAT 'WORKSTATION’::(SIMXEVELl]V.DAT 

$ IF NSUB EQ. ’P2’ THEN GOTO START4 
SCT-CT + 1 
S GOTO SEND_DATA 
$ START4: 

S COPY VS.DAT [SIMXEVEL1]VDAT 
$ GOTO START1 
$ FINISH: 

TYPE in Batch Procedure for each subsystem 


S’ 

S! 

S’ 

S! 

S’ 

S! 

S’ 


PROCEDURE RUN 11 EXECUTES LEVEL1S1 
"^WORKSTATION NAME WHERE TOPLEV RUNS 


s SET VER 

$ SET MESSAGE/NOF/NOS/NOI/NOT 

5 NOTOPEN: 

$! CHECK FOR TERMINATION FILE OUT DAT 
e p^EX , FSSEARCHCOLTr.DA'r) 

$ IF FILEX NES. THEN GOTO FINISH 
$ WAIT 00.00:05.00 
$ RUNIT: 

j, WATT FOR INPUT DATA FILES V.DAT AND LEVEL IS 1 CM 

J OPEN/READ/ERROR=NOTOPEN FILE V DAT 

$ CLOSE FILE 
$ SET MESSAGE/F/S/I/T 
S NOTOPEN2: 

S FILEX -FSSEARCHCOUT.DAT") 

$ ShLEX NFS. "THEN GOTO FINISH 
$ WAIT 00:03:05.00 

I OPEN/R£AD/ERROR=NOTOPEN2 FILE LEVEL1S1.CM 
$ CLOSE FILE 
$ RUN2: 

si RUN LEVEL 1 SUBSYSTEM 1 OPTIMIZATION CODE 

SI 

$ RUN LEVEL1S1 

SI USE DECNETTO COPY LV1I.CMTO LEVEL DATA MANAGER 

S COPY [SIM.LEVEL11LV11.CM ’PI':. [SIM. LEVEL! ]* 

$ GOTO START FINISH: 

TYPE IV Batch file to manage data for top level and 
run system optimization 


$! 

S! 

$! 

S! 

$! 

S’ 

S’ 

S’ 

S! 

$! 

S! 

S! 


PROCEDURERUNT.COM RUNS TOPLEVEL OPTIMIZATION 


P5 WORKSTATION NAME FOR BOTTOM IiVEL SUB 
P6 WORKSTATION NAME FOR BOTTOM LEVEL SUB 3 
P7 WORKSTATION NAME FOR BOTTOM LEVE 


$ SET DEF [SIM.LEVEL1] 
$ DEL • OUT;* 

$ DEL OUT .DAT;* 

$ PURGE * * 


$ ON ERROR THEN GOTO ERR 
S SET DEF [SIM.LEVEL1] 

$ DEL OUT .DAT;* 

$ PURGE *.* 

$ NUM_SYSB - P2 
S NUM_SYS1 - PI 
$ SET VER 
$ START: 

SNB1 -1 

S SET MESSAGE/NOF/NOS/NOI/NOT 

S NOTOPEN: 

S! CHECK FOR TERMINATION FILE OUT.DAT 

$ FILEX - FSSEARCHCOUT.DAT’) 

$ IF FILEX .NES. THEN GOTO STOP 
$ WAIT 00:00:05.00 

$! WAIT FOR INPUT FILE FROM LEVEL 1 SUBSYSTEMS 

j'lF(NBl.GT.NUM_SYSl) THEN GOTONEXT 
S OPEN/READ/ERROR =NOTOPEN FILE L.V1 NB1 CM 

$ CLOSE FILE 



SNBi -NB1 + 1 
$ GOTO NOTOPEN 
$ NEXT: 


J OPEN/READ/ERROR-NOTOPEN FILE V DAT 
$ CLOSE FILE 
5! 

RUN TOP LEVEL OPTIMIZATION CODE 

S! 

$ SET MESSAGE/F/S/I/T 
$ RUN TOPLEV 
S! 

FOR TERMINATION FILE OUT .DAT 

S FILEX * FSSEARCHCOUT.DA'T) 

S IF FILEX NES, w " THEN GOTO STOP 

S! SEND INPUT FILES TO BOTTOM LEVEL SUBSYSTEM! 

SCT-4 
$ REPEAT: 

S WORKSTATION * P’CT 
$ NBB - NBB + 1 
SCOPYLEVP3 S NBB .CM 

^^SIM.LEVEL’Pa’lLEV’PS’S’NBB' CM 

* ' W0 ^ ST ^^':U ”S3.fv^AT 

S IF NBB LT. NUMSYSB THEN GOTO REPEAT 
$ COPY VS .DAT [KCY.LEVEL’P3’]V DAT 
$ PURGE *SAV 
S GOTO START 
$ STOP: 


J DONE: 

$! 

S! COPY OUTPUT FILE TO PRINTER 
$! 

J PRINT TOPLEV.OUT 
$ GOTO FINISH 
SERR: 


$ COPY fSIMJERR.DAT OUT DAT 
S FINISH: 


S FILEX * F$SEARCHfOLT.DAr) 

S IF FILEX .NES. - THEN GOTO FINISH 
$ CONTINUE: 

S WAIT 00:00:10.00 
S GOTO START 
5 TIME* FSTIMEO 
S SHOW SYMBOL TIME 


S! SEND TCRVnNATO)N F[1K OtT DAT TO STOP ALL PROCESSORS 

S COPY OUT.ERR OUT.DAT 
i FINISH: 

l OLrr DAT WORKSTATIONS: : [SIM LEVELUOIT DAT 

SCOPY OUT.DAT WORKSTA-nON’PS-: [SIMLEVELllOLTnAT 

* COPY OUTDAT WORKSTATIONS (SIM LEVEL] IOCT DAT 

* ™PY OUTJJAT WORKSTATION^-:: [SIM LEWuloLT DAT 

sssisir*™" 

J^SUSJECr^PROCESSESSHLTOOWN™ OLT DAT SIM 


Type V Batch Procedure that acts as time checker 
and system terminator 

U* T m “ Sl * “ “K dfowory as 

tne TOPLEVEL procedure file 

$! 

S! 

$! 

5 ! 

S! 

$f 
S! 

$! 

S! 

Sf 


procedure TIMECK.COM 
INPUT 

PI - TIME LIMIT IN SECONDS 
P2 - NODE NAME FOR OTHER PROCESSOR 
P3 - NODE NAME FOR OTHER PROCESSOR 
P4 - NODE NAME FOR OTHER PROCESSOR 
P5 - NODE NAME FOR OTHER PROCESSOR 


| ON ERROR THEN GOTO STOP 

5 SET VER 
S TIME - FSTIME ( ) 

S TIMEL * Pi 
STIMEC-0 

J SETDEF [SIM.LEVELII 

S! CHECK TIME 
J! 

S START: 

J TIMEC » TIMEC + 10 
$ IF TIMEC GT TIMEL THEN GOTO STOP 


$! CHECK FOR TERMINATION FILE GENERATED BY TOPLEV 
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